Nodes can now spawn an HTTP server that is meant to receive chunked files as a stream, without actually having to send the proofs for each chunk. The endpoint is meant only for "trusted" sources that would validate the files are correct before sending them (Since validation happens after storing all chunks, whoever can hit the endpoint can DoS the MSP by sending lots of files).
The endpoint is now used by the backend, replacing the rpc/p2p dedicated logic that handled the sending of proven chunks for files.

⚠️ Breaking Changes ⚠️

• Short description

1. Added msp_trusted_file_transfer_server_url argument to the backend. Backend now needs access to this endpoint for uploads to work properly.
2. Added trusted_file_transfer_server, trusted_file_transfer_server_host and trusted_file_transfer_server_port options for the storage hub node to spawn the server (Warning: The server is not meant to be exposed to the public, it should only be reachable to the backend servers via firewall rules)

• Who is affected
  MSP providers who expose a backend endpoint.

• Suggested code changes

1. Set up MSP node so that it spawns the http file transfer server on the desired host/port.
2. Configure backend server's msp_trusted_file_transfer_server_url accordingly.
3. Set up firewall rules so that only backend servers can establish a connection with the trusted file transfer server.

Further Changes

1. Doing concurrent writes to the same file key leads to unexpected behavior (Both calls will end up in a FileIncomplete error). This is probably better handled at the file_storage level. For now the backend gatekeeps concurrent writes via a guard (see fix(backend): 🐛 avoid concurrent uploads for the same file key #586)
2. If chunk by chunk writes are found to be a bottleneck, batch writing of chunks could be implemented.
3. Right now there is no handling of errors in the way that it is done with the p2p upload (Where file is rejected on certain errors). It probably does not make sense to reject the file, but we need to make sure the storage is left in a good state where the file upload can be reattempted again.
4. Consider having a trusted file transfer client in the backend so the tcp connection is reused.

Extra Notes

1. Decided to remove upload_to_msp backend test that uses mocks because it required restructuring things just for the connection with the http server to be mockable and the flow is already covered by both the backend integration tests and the process_chunk_stream tests. Adding a mocked test adds little to no value for the extra complexity imo.
2. Had to split encoding/decoding (Which should have a single source of truth) from the server logic and put it in shc-common because adding shc-client as a backend dependency caused many conflicts due to it being compiled with no-std. I was not able to find a trivial way to propagate the std feature on the required packages.